Using a Three Dimensional Spatial Database to Orthorectify Automatically Remote Sensing Images

Following SPOT5 launch, Spot Image and French National Geographic Institute (IGN) have decided to design a worldwide accurate database called Reference3DTM using data from the High Resolution Stereoscopic SPOT5 instrument (HRS). This database consists in three information layers : Digital Elevation Model at 1 arc.second resolution, Orthoimage at 1/6th arc.second resolution and Quality Masks ; with a specification of circular planimetric accuracy better than 16 m for 90 % of the points and elevation accuracy better than 10 m for 90% of the points. It has also been decided to develop and commercialize a system called ANDORRE to archive Reference3D database and produce orthorectified images and DEM thanks to Reference3DTM data. ANDORRE has been designed to take advantage of Reference3D horizontal and altimetric accuracy to automatically register and rectify any image from SPOT satellites in any area where Reference3DTM data is available. This system could also be upgraded to process other satellite data. Spot Image has delegated to French Space Agency (CNES) the realization and industrialization of the algorithmic core of the processing chain. This paper presents the automatic ortho-rectification process composed of three main steps : generation of a reference image (in focal plane geometry) using SPOT rigorous sensor model and Reference3DTM Orthoimage and DEM layers ; modeling of the mis-registration between the reference image and the SPOT image to be processed, using a multiresolution image matching algorithm, in order to improve the SPOT location model;resampling of the image into a cartographic reference frame using Reference3DTM DEM and the improved SPOT location model.It also focuses on some difficult cases and the robustness of the algorithm (different landscapes, DEM and image resolution) and on the processing time needed to achieve the rectification of the SPOT image in different cases. 1. OVERVIEW Ortho-rectification of remote sensing images is an important issue for various applications. Indeed, the image orthorectification process combines relief effects corrections and geo-referencing with high location accuracy. Usually this process is supervised by an operator with a manual selection of ground control points (using maps or reference images) which is fastidious and may lead to variable qualities of ortho-rectification. Reference3DTM is a three dimensional database which has been designed to take advantage of SPOT5 stereoscopic images high accuracy. This world wide database consists in three layers : a Digital Terrain Model (1 arc.second resolution, DTED level 2 standard), an Orthoimage (1/6th arc.second resolution) and Quality Masks ; with a specification of circular horizontal accuracy better than 16 m for 90 % of the points and elevation accuracy better than 10 m for 90% of the points (Airault et.al., 2003). The ANDORRE system is an operational system that leverages this database to automatically ortho-rectify SPOT images (Figure 1). The algorithmic core of ANDORRE is called TARIFA (French acronym for Automatic Image Rectification and Fusion Processing). This paper presents the concepts and the results of the operational version of TARIFA. Figure 1. Reference3DTM Database and ANDORRE SPOT OrthoImage SPOT Scene 1A ANDORRE system OrthoImage Reference3D ® database DEM Automatic location Relief correction Image processingcore(TARIFA) ISPRS Workshop on Service and Application of Spatial Data Infrastructure, XXXVI (4/W6), Oct.14-16, Hangzhou, China 90 2. DESCRIPTION OF THE PROCESSING METHOD Since the beginning of the definition of Reference3DTM database concept, the idea to take advantage of it in order to automatically rectify SPOT images has been developed. A preliminary study was early held at French National Geographic Institute Space Department. The output of this study was a definition of an automatic orthorectification process and a prototype. This prototype had been tested against 50 different SPOT Images (from SPOT1 to SPOT4) and some simulated Reference3DTM tiles (Mangolini et.al., 2002). After SPOT5 launch, and first Reference3DTM tiles available, validation of the processing using SPOT5 images and real Reference3DTM data was necessary. The algorithm parameters had also to be tuned. Thus, a new version of the prototype was developed and tested by CNES. Finally, the operational version of the algorithm was developed with CNES as prime contractor to be in integrated by Spot Image in its ground segment (ANDORRE). The automatic rectification process is mainly based on the improvement of the geometric model using image matching measures between Reference3DTM data and SPOT images. The algorithm is divided in four stages: 2.1 Resampling of Reference3DTM into SPOT geometry This first stage consists in simulating an image from Reference3DTM database of the landscape seen by the satellite during the acquisition of the SPOT scene. Thus, Reference3DTM tiles covering the SPOT scene area are first assembled in order to obtain a single orthoimage and DTM corresponding to the SPOT scene. The rigorous sensor model of the SPOT scene is then used with Reference3DTM DTM layer to compute a resampling model from ground to raw scene geometry (using a ray tracing method which takes into account the DTM variations). The orthoimage layer extracted from Reference3DTM is resampled to obtain an image as seen by the satellite called the simulated image (see Figure 3). Figure 3. Computing simulated image resampling grid Thus, the remaining misregistration between simulated image and Spot scene is only due to imprecision of the SPOT location model (Bouillon et.al., 2003). 2.2 Determination of the Correction Model by Multiresolution Image Matching In this stage, image matching measures are computed between the SPOT scene and the simulated image in order to obtain a correction model which will allow to match the SPOT scene with the simulated image (see Figure 4). First, different zoom levels, with a ratio of 2 between different levels, are created for the two images according to the expected location accuracy. The initial zoom level is computed in accordance with (1). The goal is to keep a searching window for the image matching process of 5 pixel size. Figure 4. Correction model estimation by multiresolution image matching Use of the SPOT location model associated to the SPOT scene taking into account the Reference3DTM sensor geometry ground geometry Image Matching Tie point Selection & Homogenei-sation Correction Model Adjustment Images Dezoom Final Correction Model For each zoom level Convergence critera (full resolution) Current correction model Correlation grid Tie Point list Resampling of the simulated image: Reference3DTM image in SPOT Reference3DTM in « ground geometry » Use of the SPOT location model associated to the SPOT scene taking into account the Reference3DTM DTM Figure 2 . Resampling of Reference3DTM tiles into “raw sensor geometry” ISPRS Workshop on Service and Application of Spatial Data Infrastructure, XXXVI (4/W6), Oct.14-16, Hangzhou, China 91 ize WinSearchS ution SceneResol y LocAccurac t MinZoomIni × = (1) For example, for SPOT4 10 m mode, an initial zoom level of 64 is needed. For each zoom level, an automatic image matching is performed. The matching is based on correlation coefficient computation and its main parameters are the correlation window size and the correlation validity threshold. The result of the automatic matching is a grid of measured differences between the two images. This grid is filtered in order to select tie points. Finally, the collected tie points are used to calibrate a correction model of the SPOT scene. The adjustment process includes a statistical filtering of bad points (K standard deviation filter). The model estimation depends on the zoom level. We have defined 3 phases: a bias is estimated only for first zoom levels (phase 1), then a polynomial model (degree 1 or 2) is estimated (phase 2), and at the full resolution, some matching iterations are made in order to refine the correction model (phase 3 or refining phase). At each zoom level, the model previously estimated is taken into account in the matching process to predict the differences between the two images and keep a 5 pixel searching window size. In full resolution (refining phase), the convergence criteria is the difference between current and previous model and is processed for a grid of control points chosen in the image. If the difference becomes lower than a convergence threshold (usually <50cm), the model estimation is finished. 2.3 Rectification Grid Processing In this stage, the rectification grid for the SPOT scene is computed by combination of the correction model previously computed, a reverse location grid in the geographic reference system WGS84 obtained by the rigorous sensor model of the SPOT image, and a conversion grid from WGS84 reference system to the cartographic system required in output. Here, the main point is to take advantage of the full resolution of Reference3DTM DTM to produce the reverse location grid with the best accuracy possible and a minimum computing time. Several methods have been studied to process this grid and are detailed in(Baillarin et.al., 2004)]. The principle of the choosen method is to compute the physical reverse location model on two grids of constant altitude (H1 and H2) with a large step, fill those grids by spatial interpolation and obtain the final dense grid by altitude interpolation. For a point (X, Y, H) the image location is approximated by: • bi-linear interpolation of (X, Y, H1) in the first grid, • bi-linear interpolation of (X, Y, H2) in the second grid, • linear interpolation of (X, Y, H) between (X, Y, H1) and (X, Y, H2). Figure 5. Interpolated reverse model principle This method is based on the hypothesis that the reverse location is linearly dependant on both altitude and planimetry. The determination of H1 and H2 is important to ensure this linearity. Taking the minimum and maximum altitude value on the concerned DTM is the optimal choice. 2.4 Resampling of the SPOT Scene This last stage consists in a resampling of the SPOT scene using the rectification grid processed at the previous stage. In addition to the image rectification, all the tie points finally used to compute the correction model are converted into ground control points using Reference3DTM location. These points are kept for use in case of another processing of the same image or an image from the same datastrip. 3. OPERATIONAL VALIDATION OF THE PROCESSING A first validation has been done step by step on the prototype version in order to tune and optimize the algorithm(Baillarin et.al., 2004). Then, an operational qualification of the system has been achieved on various cases. In this paragraph, we analyse some difficult cases and the robustness of the algorithm. 3.1 Tuning of theImage Matching Algorithm and Tie Point Filtering Method Different problems have been observed due to low resemblance between images. Figure 6 gives an example of seasonal variations over rice field landscapes in China. In such cases, the image matching algorithm based on correlation coefficients computation doesn’t find enough valid tie points between the two images. The correlation coefficients are smaller than in other cases and do not reach the validity threshold (80%). Therefore in order to find enough tie points in all cases, a lower validity threshold must be selected (70% instead of 80% of resemblance). Therefore in order to find enough tie points in all cases, a lower validity threshold must be selected (70% instead of 80% of resemblance). dh2